Continuity of mutual entropy in the large signal-to-noise ratio limit

The aim of this paper is to analyse continuity properties of mutual and conditional entropies, between the input and output of a channel with additive noise. Our attention is focused mainly on a distinctly non-Gaussian situation, for both large and small signal-to-noise ratio. To our knowledge, this nontrivial aspect has not been discussed before at the level of generality adopted in this paper. A complex character of the continuity properties of various entropies was acknowledged as early as in the 1950’s; see, e.g., paper [1] where a number of important (and elegant) results have been proven, about limiting behaviour of various entropies. An additional motivation was provided by the short note [9] suggesting an elegant method of deriving the so-called entropy-power inequality (EPI). The way of reasoning in [9] is often referred to as the direct probabilistic method, as opposite to the socalled analytic method; see [6], [5], [2], [8]. The results of this paper (Lemmas 2.1– 2.4 and Lemma 3.1) provide additional insight on the assumptions under which the direct probabilistic method can be used to establish the EPI in a rigorous manner. For completeness, we give in Section 4 a short derivation of the EPI in which we follow Ref. [9] but clarify a couple of steps thanks to our continuity lemmas. However, without rigorously proven continuity properties of mutual and conditional entropies in both signal-to-noise ratio regimes, the derivation of the EPI via the direct probabilistic method cannot be accomplished. Another approach to EPI, for discrete random variables (RVs) where it takes a different form, is discussed in [4], see also references therein. For the history of the question, see [2]; for reader’s convenience, the statement of the EPI is given at the end of this section.